No-audio speaking status detection in crowded settings via visual pose-based filtering and wearable acceleration

Recognizing who is speaking in a crowded scene is a key challenge towards the understanding of the social interactions going on within. Detecting speaking status from body movement alone opens the door for the analysis of social scenes in which personal audio is not obtainable. Video and wearable sensors make it possible recognize speaking in an unobtrusive, privacy-preserving way. When considering the video modality, in action recognition problems, a bounding box is traditionally used to localize and segment out the target subject, to then recognize the action taking place within it. However, cross-contamination, occlusion, and the articulated nature of the human body, make this approach challenging in a crowded scene. Here, we leverage articulated body poses for subject localization and in the subsequent speech detection stage. We show that the selection of local features around pose keypoints has a positive effect on generalization performance while also significantly reducing the number of local features considered, making for a more efficient method. Using two in-the-wild datasets with different viewpoints of subjects, we investigate the role of cross-contamination in this effect. We additionally make use of acceleration measured through wearable sensors for the same task, and present a multimodal approach combining both methods

Impact of annotation modality on label quality and model performance in the automatic assessment of laughter in-the-wild

Laughter is considered one of the most overt signals of joy. Laughter is well-recognized as a multimodal phenomenon but is most commonly detected by sensing the sound of laughter. It is unclear how perception and annotation of laughter differ when annotated from other modalities like video, via the body movements of laughter. In this paper we take a first step in this direction by asking if and how well laughter can be annotated when only audio, only video (containing full body movement information) or audiovisual modalities are available to annotators. We ask whether annotations of laughter are congruent across modalities, and compare the effect that labeling modality has on machine learning model performance. We compare annotations and models for laughter detection, intensity estimation, and segmentation, three tasks common in previous studies of laughter. Our analysis of more than 4000 annotations acquired from 48 annotators revealed evidence for incongruity in the perception of laughter, and its intensity between modalities. Further analysis of annotations against consolidated audiovisual reference annotations revealed that recall was lower on average for video when compared to the audio condition, but tended to increase with the intensity of the laughter samples. Our machine learning experiments compared the performance of state-of-the-art unimodal (audio-based, video-based and acceleration-based) and multi-modal models for different combinations of input modalities, training label modality, and testing label modality. Models with video and acceleration inputs had similar performance regardless of training label modality, suggesting that it may be entirely appropriate to train models for laughter detection from body movements using video-acquired labels, despite their lower inter-rater agreement

ConfLab: A Rich Multimodal Multisensor Dataset of Free-Standing Social Interactions in the Wild

Recording the dynamics of unscripted human interactions in the wild is challenging due to the delicate trade-offs between several factors: participant privacy, ecological validity, data fidelity, and logistical overheads. To address these, following a 'datasets for the community by the community' ethos, we propose the Conference Living Lab (ConfLab): a new concept for multimodal multisensor data collection of in-the-wild free-standing social conversations. For the first instantiation of ConfLab described here, we organized a real-life professional networking event at a major international conference. Involving 48 conference attendees, the dataset captures a diverse mix of status, acquaintance, and networking motivations. Our capture setup improves upon the data fidelity of prior in-the-wild datasets while retaining privacy sensitivity: 8 videos (1920x1080, 60 fps) from a non-invasive overhead view, and custom wearable sensors with onboard recording of body motion (full 9-axis IMU), privacy-preserving low-frequency audio (1250 Hz), and Bluetooth-based proximity. Additionally, we developed custom solutions for distributed hardware synchronization at acquisition, and time-efficient continuous annotation of body keypoints and actions at high sampling rates. Our benchmarks showcase some of the open research tasks related to in-the-wild privacy-preserving social data analysis: keypoints detection from overhead camera views, skeleton-based no-audio speaker detection, and F-formation detection.Comment: v2 is the version submitted to Neurips 2022 Datasets and Benchmarks Trac

Imagens aéreas multiespectrais para fenotipagem e contagem de plantas: estudos de caso em ervilha (Pisum sativum) e viveiro de maçã (Malus domestica)

Field data collection involves time and money consuming processes, additionally carrying possible measurement errors. With the technological advance in the last years, low cost remote sensing tools have emerged to facilitate procedures for in-field measurements, being one of the most known techniques the use of multispectral cameras coupled to RPA. These tools are complemented by the implementation of procedures in GIS and image-processing software, from which are developed methodologies leading to extract target values from a certain original set of data. In this work, multispectral images were used in two case studies: (1) for yield estimation in pea plots for breeding research, and (2) for plant counting in an apple nursery planted directly on the soil; both fields are located in Washington State, USA. In the first case, a reliable and replicable methodology for yield estimation was created as a high throughput phenotyping technique; while in the second case an algorithm capable of identifying the number of apple plants with more than 95% accuracy was developed. In both studies, remote sensing is used as an efficient and practical way to improve field operations under the specified conditions of each case.A coleta de dados de campo envolve processos de grande consumo em tempo e dinheiro, ademais de levar o risco de possíveis erros de medição. Com o avanço tecnológico nos últimos anos, surgiram ferramentas de sensoriamento remoto de baixo custo para facilitar procedimentos de medição em campo, sendo uma das técnicas mais conhecidas o uso de câmeras multiespectrales acopladas a um ARP. Essas ferramentas são complementadas pela implementação de procedimentos em programas SIG e de processamento de imagens, a partir dos quais são desenvolvidas metodologias que visam extrair valores alvo desde um determinado conjunto original de dados. Neste trabalho, foram utilizadas imagens multiespectrais no desenvolvimento de dois estudos de caso: (1) para estimativa de produtividade em parcelas para pesquisa de ervilha, e (2) para contagem de plantas em um viveiro de maçã plantado diretamente no solo; ambos os campos localizados no estado de Washington, EUA. No primeiro caso, foi criada uma metodologia confiável e replicável para estimativa de produtividade como técnica de fenotipagem de alto rendimento; enquanto no segundo caso, foi desenvolvido um algoritmo capaz de identificar o número de plantas de maçã com mais de 95% de exatidão. Em ambos os estudos, o sensoriamento remoto é usado como uma ferramenta eficiente e prática na melhora de operações de campo

Imagens aéreas multiespectrais para fenotipagem e contagem de plantas: estudos de caso em ervilha (Pisum sativum) e viveiro de maçã (Malus domestica)

Field data collection involves time and money consuming processes, additionally carrying possible measurement errors. With the technological advance in the last years, low cost remote sensing tools have emerged to facilitate procedures for in-field measurements, being one of the most known techniques the use of multispectral cameras coupled to RPA. These tools are complemented by the implementation of procedures in GIS and image-processing software, from which are developed methodologies leading to extract target values from a certain original set of data. In this work, multispectral images were used in two case studies: (1) for yield estimation in pea plots for breeding research, and (2) for plant counting in an apple nursery planted directly on the soil; both fields are located in Washington State, USA. In the first case, a reliable and replicable methodology for yield estimation was created as a high throughput phenotyping technique; while in the second case an algorithm capable of identifying the number of apple plants with more than 95% accuracy was developed. In both studies, remote sensing is used as an efficient and practical way to improve field operations under the specified conditions of each case.A coleta de dados de campo envolve processos de grande consumo em tempo e dinheiro, ademais de levar o risco de possíveis erros de medição. Com o avanço tecnológico nos últimos anos, surgiram ferramentas de sensoriamento remoto de baixo custo para facilitar procedimentos de medição em campo, sendo uma das técnicas mais conhecidas o uso de câmeras multiespectrales acopladas a um ARP. Essas ferramentas são complementadas pela implementação de procedimentos em programas SIG e de processamento de imagens, a partir dos quais são desenvolvidas metodologias que visam extrair valores alvo desde um determinado conjunto original de dados. Neste trabalho, foram utilizadas imagens multiespectrais no desenvolvimento de dois estudos de caso: (1) para estimativa de produtividade em parcelas para pesquisa de ervilha, e (2) para contagem de plantas em um viveiro de maçã plantado diretamente no solo; ambos os campos localizados no estado de Washington, EUA. No primeiro caso, foi criada uma metodologia confiável e replicável para estimativa de produtividade como técnica de fenotipagem de alto rendimento; enquanto no segundo caso, foi desenvolvido um algoritmo capaz de identificar o número de plantas de maçã com mais de 95% de exatidão. Em ambos os estudos, o sensoriamento remoto é usado como uma ferramenta eficiente e prática na melhora de operações de campo

DNA methylome analysis identifies accelerated epigenetic ageing associated with postmenopausal breast cancer susceptibility

Aim of the study: A vast majority of human malignancies are associated with ageing, and age is a strong predictor of cancer risk. Recently, DNA methylation-based marker of ageing, known as ‘epigenetic clock’, has been linked with cancer risk factors. This study aimed to evaluate whether the epigenetic clock is associated with breast cancer risk susceptibility and to identify potential epigenetics-based biomarkers for risk stratification. Methods: Here, we profiled DNA methylation changes in a nested case control study embedded in the European Prospective Investigation into Cancer and Nutrition (EPIC) cohort (n = 960) using the Illumina HumanMethylation 450K BeadChip arrays and used the Horvath age estimation method to calculate epigenetic age for these samples. Intrinsic epigenetic age acceleration (IEAA) was estimated as the residuals by regressing epigenetic age on chronological age. Results: We observed an association between IEAA and breast cancer risk (OR, 1.04; 95% CI, 1.007-1.076, P = 0.016). One unit increase in IEAA was associated with a 4% increased odds of developing breast cancer (OR, 1.04; 95% CI, 1.007-1.076). Stratified analysis based on menopausal status revealed that IEAA was associated with development of postmenopausal breast cancers (OR, 1.07; 95% CI, 1.020-1.11, P = 0.003). In addition, methylome-wide analyses revealed that a higher mean DNA methylation at cytosine-phosphate-guanine (CpG) islands was associated with increased risk of breast cancer development (OR per 1 SD = 1.20; 95 %CI: 1.03-1.40, P = 0.02) whereas mean methylation levels at non-island CpGs were indistinguishable between cancer cases and controls. Conclusion: Epigenetic age acceleration and CpG island methylation have a weak, but statistically significant, association with breast cancer susceptibility. (C) 2017 Elsevier Ltd. All rights reserved

Human immunodeficiency virus continuum of care in 11 european union countries at the end of 2016 overall and by key population: Have we made progress?

Background. High uptake of antiretroviral treatment (ART) is essential to reduce human immunodeficiency virus (HIV) transmission and related mortality; however, gaps in care exist. We aimed to construct the continuum of HIV care (CoC) in 2016 in 11 European Union (EU) countries, overall and by key population and sex. To estimate progress toward the Joint United Nations Programme on HIV/AIDS (UNAIDS) 90-90-90 target, we compared 2016 to 2013 estimates for the same countries, representing 73% of the population in the region. Methods. A CoC with the following 4 stages was constructed: number of people living with HIV (PLHIV); proportion of PLHIV diagnosed; proportion of those diagnosed who ever initiated ART; and proportion of those ever treated who achieved viral suppression at their last visit. Results. We estimated that 87% of PLHIV were diagnosed; 92% of those diagnosed had ever initiated ART; and 91% of those ever on ART, or 73% of all PLHIV, were virally suppressed. Corresponding figures for men having sex with men were: 86%, 93%, 93%, 74%; for people who inject drugs: 94%, 88%, 85%, 70%; and for heterosexuals: 86%, 92%, 91%, 72%. The proportion suppressed of all PLHIV ranged from 59% to 86% across countries. Conclusions. The EU is close to the 90-90-90 target and achieved the UNAIDS target of 73% of all PLHIV virally suppressed, significant progress since 2013 when 60% of all PLHIV were virally suppressed. Strengthening of testing programs and treatment support, along with prevention interventions, are needed to achieve HIV epidemic control